Papers with QA task
A Graph Representation of Semi-structured Data for Web Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies treat semi-structured data as flat documents with pieces of text . semi-structural data is more effective to represent rich relational information . question answering is an important feature in most search engines . |
| Approach: | They propose a graph representation of Web tables and lists based on categorization of components and their relations . they also develop reasoning techniques on the graph model for the question answering task . |
| Outcome: | The proposed graph improves F1 score by 3.90 points over the state-of-the-art baselines on real datasets. |
PRover: Proof Generation for Interpretable Reasoning over Rules (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that transformers can act as “soft theorem provers” by answering questions over explicitly provided knowledge in natural language. |
| Approach: | They propose a transformer-based model that answers binary questions over rule-bases and generates the corresponding proofs. |
| Outcome: | The proposed model generates proofs with an accuracy of 87% while maintaining or improving performance on the QA task. |
How Well Do Multi-hop Reading Comprehension Models Understand Date Information? (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing multi-hop reading comprehension datasets have reasoning shortcuts that can be used to answer comparison questions without performing multi- hop reasoning. |
| Approach: | They propose a dataset with three probing tasks in addition to the main question . they then evaluate the model's ability to understand date information . |
| Outcome: | The proposed model performs well in date comparison and number subtraction tasks. |
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Observable changes in the scene are reflected in captions, but actions are also linked to social aspects such as intentions, effects, and attributes that describe the agent. |
| Approach: | They propose to generate captions from videos that describe latent aspects of the human agent's actions. |
| Outcome: | The proposed model can be used to describe latent aspects of human actions in video clips and answer questions about videos. |
Linguistically Conditioned Semantic Textual Similarity (2024.acl-long)
Copied to clipboard
| Challenge: | Semantic textual similarity (STS) is a fundamental NLP task that measures the semantic similarity between two sentences. |
| Approach: | They propose to use a conditional STS dataset to measure sentences’ similarity conditioned on a certain aspect to reduce the inherent ambiguity posed by the sentences. |
| Outcome: | The proposed method improves the performance over baselines on the C-STS dataset with over 80% F1 score. |
PolicyQA: A Reading Comprehension Dataset for Privacy Policies (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Privacy policy documents are long and verbose. Hence, a question answering system can help users find the information that is relevant and important to them. |
| Approach: | They propose to provide users with a short text span from policy documents to search for answers from a long text segment. |
| Outcome: | The proposed question answering system can help users find information relevant to them. |
SEMQA: Semi-Extractive Multi-Source Question Answering (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent proposed long-form question answering systems have shown promising capabilities, but attributing and verifying their generated abstractive answers can be difficult. |
| Approach: | They propose a task that summarises multiple sources in a semi-extractive fashion . they create a dataset with human-written semi-extractive answers to natural and generated questions . |
| Outcome: | The proposed task summarizes multiple sources in a semi-extractive fashion and produces fine in-line attributions by-design that are easy to verify, interpret, and evaluate. |
Yes, No or IDK: The Challenge of Unanswerable Yes/No Questions (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for Yes/No QA are lacking information needed to answer a Yes/Non question. |
| Approach: | They extend the Yes/No QA task by adding questions with an IDK answer to a BoolQ dataset and create out-of-domain test sets for the task. |
| Outcome: | The proposed dataset includes paragraphs together with naturally occurring questions whose answer is either "Yes" or "No". |
Analyzing the Effectiveness of the Underlying Reasoning Tasks in Multi-hop Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies have utilized underlying reasoning (UR) tasks in multi-hop question answering datasets to explain the predicted answers and evaluate models' reasoning abilities. |
| Approach: | They analyze UR tasks in QA datasets to determine their effectiveness . they find that UR task is helpful in preventing reasoning shortcuts . |
| Outcome: | The proposed model improves QA performance, reasoning shortcuts, and robustness on adversarial questions. |
Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses (2025.emnlp-industry)
Copied to clipboard
| Challenge: | DeCE is model-agnostic and domain-general, requiring no predefined taxonomies or handcrafted rubrics. |
| Approach: | They propose a decomposed LLM evaluation framework that separates accuracy and recall from accuracy and relevance. |
| Outcome: | The proposed framework achieves stronger correlation with expert judgments than traditional metrics and pointwise LLM scoring. |
A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have enabled advances in the field of natural language processing . however, their application and potential are still underexplored . |
| Approach: | They evaluate four state-of-the-art instruction-tuned Large Language Models on 13 NLP tasks in English. |
| Outcome: | The evaluated models outperform state-of-the-art models on 13 real-world clinical and biomedical NLP tasks in English. |
Characterizing LLM Abstention Behavior in Science QA with Context Perturbations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has investigated the ability of LLMs to abstain from answering context-dependent questions when provided insufficient or inconsistent context is provided. |
| Approach: | They propose to improve abstention when provided insufficient or incorrect context . they probed the ability of LLMs to abstain from answering context-dependent science questions . |
| Outcome: | The proposed models abstain from answering science questions when provided insufficient or incorrect context. |
Bi-directional CognitiveThinking Network for Machine Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for reading comprehension are still in their infancy at the level of cognitive intelligence. |
| Approach: | They propose a bi-directional cognitive knowledge framework to simulate reverse thinking and inertial thinking in the brain to answer questions. |
| Outcome: | The proposed framework shows that bi-directional knowledge helps the QA task. |
Mitigating Bias for Question Answering Models by Tracking Bias Influence (2024.naacl-long)
Copied to clipboard
Mingyu Ma, Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Tagyoung Chung, Wei Wang, Kai-Wei Chang, Nanyun Peng
| Challenge: | Existing literature observes bias in question answering (QA) models, but there is no method to mitigate it. |
| Approach: | They propose an approach to mitigate the bias of question answering models by observing the influence of a query instance on another instance. |
| Outcome: | The proposed method reduces bias level in all 9 bias categories while maintaining comparable QA accuracy. |
ProQA: Structural Prompt-based Pre-training for Unified Question Answering (2022.naacl-main)
Copied to clipboard
Wanjun Zhong, Yifan Gao, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan
| Challenge: | Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills. |
| Approach: | They propose a unified QA paradigm that solves various tasks through a single model. |
| Outcome: | The proposed model improves QA-centric ability on 11 QA benchmarks. |
Using Natural Language Relations between Answer Choices for Machine Comprehension (N19-1)
Copied to clipboard
| Challenge: | Current approaches to the reading comprehension task quantify the relationship between each question and answer choice independently and pick the highest scoring option. |
| Approach: | They propose a method to leverage natural language relations between answer choices to improve machine comprehension. |
| Outcome: | The proposed model improves the performance of a reading comprehension task by leveraging natural language relations between answer choices. |
Understanding Dataset Design Choices for Multi-hop Reasoning (N19-1)
Copied to clipboard
| Challenge: | Existing datasets that explicitly focus on multi-hop reasoning are lacking in learning multi-tasking. |
| Approach: | They propose to use sentence-factored models to solve multi-hop question answering tasks . they find spurious correlations in unmasked versions of WikiHop and HotpotQA . |
| Outcome: | The proposed datasets are used to test models on multi-hop question answering tasks. |
Learning to Perturb Word Embeddings for Out-of-distribution QA (2021.acl-long)
Copied to clipboard
| Challenge: | QA models that are pretraining with unlabeled data can overfit and may not generalize well to unseen data that falls outside the training distribution. |
| Approach: | They propose a method which perturbs word embedding without changing their semantics. |
| Outcome: | The proposed method outperforms baseline methods on five target domains on a single source dataset on five different target domain domains. |
Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding (2024.emnlp-main)
Copied to clipboard
Chong Zhang, Yi Tu, Yixi Zhao, Chenshu Yuan, Huan Chen, Yue Zhang, Mingxu Chai, Ya Guo, Huijia Zhu, Qi Zhang, Tao Gui
| Challenge: | Existing models of layout reading order do not convey the complete reading order information in the layout. |
| Approach: | They propose to model layout reading order as ordering relations over layout elements . they propose a reading-order-relation-enhancing pipeline to improve model performance . |
| Outcome: | The proposed model outperforms existing models on a visual-rich document dataset and on eight cross-domain VrD-IE/QA tasks without targeted optimization. |
More Bang for your Context: Virtual Documents for Question Answering over Long Documents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle to utilize long contexts efficiently, resulting in a question answering problem. |
| Approach: | They propose a method to generate a short document that contains the most relevant parts for a given context window. |
| Outcome: | The proposed method improves the QA task by providing a short and focused VDoc to the LLM while keeping the context window full. |
DynamicER: Resolving Emerging Mentions to Dynamic Entities for RAG (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing entity linking models struggle to link new expressions to entities in the dynamic nature of human language. |
| Approach: | They propose a task to resolve emerging mentions to dynamic entities and a benchmark to evaluate their model's adaptability to new expressions. |
| Outcome: | The proposed method outperforms baselines on QA task with resolved mentions and improves retrieval-augmented generation performance. |
JEMHopQA: Dataset for Japanese Explainable Multi-Hop Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | a dataset for explainable QA in Japanese is available for many languages, but not in other languages. |
| Approach: | They present a multi-hop QA dataset based on Japanese Wikipedia . it includes question-answer pairs and supporting evidence in the form of derivation triples . they show that the dataset is sufficiently challenging for state-of-the-art LLMs based upon this dataset . |
| Outcome: | The proposed dataset is based on Japanese Wikipedia and can be used to evaluate QA tasks. |
You Only Use Reactive Attention Slice When Retrieving From Long Context (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrieval techniques for language models are limited due to their reliance on lexical similarity and are computationally expensive to train. |
| Approach: | They propose a training-free and fine-tuning-free attention-based retrieval technique that uses a reaction score heuristic to quantify how an LM’s self-attention “reacts” to a user query. |
| Outcome: | The proposed approach improves QA task accuracy by 15% and inference throughput by 31% compared to embedding-based retrieval. |